Learning Specific Detectors of Adverse Events in Multivariate Time Series

OBJECTIVE This paper describes how powerful detectors of adverse events manifested in multivariate series of biosurveillance data can be learned using only a few labeled instances of such events. BACKGROUND The context of the work presented here is rapid detection of statistically significant emerging patterns of adverse events in data related to food-and agriculturesafety collected by the U.S. Department of Agriculture. The particular data under consideration includes records of daily counts of condemned and healthy cattle, counts of positive and negative microbial tests of food samples, and counts of passed and failed sanitary inspections of slaughter houses. Effective monitoring of those streams of heterogeneous data is instrumental in early detection of adverse food events and in their subsequent mitigation. METHOD We use temporal scan [2] as a basic detection tool. It slides a fixed-width time window along a time series and compares the positive and negative counts inside of it against the aggregated counts observed outside, and applies either Chi-square or Fisher’s exact test of significance of the obtained contingency table. In our approach, temporal scan is being applied individually to each of the available data-streams and the resulting series of p-values are then combined using Fisher’s method of p-values aggregation [3]. Detectors based on Fisher’s method benefit multivariate analysis by being able to raise an alert even if none of the component signals is critical, but if some of them are near critical. They are non-specific because they target any departure of the combined series from normal, and they are not tailored to any specific scenario.